1. Field of the Invention
This invention relates to computational photography, and in particular to enhancing visible photograph visual quality using corresponding near infra-red images.
2. Background of the Invention
The radiance from natural scenes usually spans a very wide dynamic range, far exceeding what a digital camera can capture. For instance, in a sunny outdoor environment, the dynamic range could reach as high as 109. In contrast, a professional-grade digital camera that uses 14 bits per color channel can capture a limited dynamic range of only 104. Consumer-grade cameras are even worse. There is no single exposure in cameras that can capture all the details in the brightest and darkest regions simultaneously. As a result, images/photographs of a scene captured by a digital camera omit a lot of scene details. By comparison, human eyes have a much higher dynamic range than a camera, and can perceive much more scene details omitted by a digital camera.
One conventional solution to this problem is tone mapping, which computes a high dynamic range (HDR) image, usually from multiple shots of varying exposures, and then maps the HDR image into a lower dynamic range (LDR) image suitable for display devices. However, this tone mapping technique does not usually produce a perceptually pleasing result. Usually, pixels end up becoming too bright or too dark, and rich scene information such as color and texture are almost completely lost. Furthermore, conventional tone mapping techniques require obtaining an HDR image from multiple images captured with different exposures. This HDR image requirement limits the tone mapping technique to static scenes, which greatly reduces its usefulness in everyday photography.
Another conventional solution widely used by professional photographers is to take photos in raw format and manually adjust contrast region by region. Usually raw pictures use 12 or 14 bits per color channel to record scene radiance, thus resulting in a higher dynamic range than normal Joint Photographic Experts Group (JPEG) photos. Such manual adjustment is tedious and requires experience, and the dynamic range of raw format is still very limited compared to the dynamic range human eyes can perceive.
Near Infra-Red (NIR) light lies between visible red light and Long Infra-Red (LIR) light in the electromagnetic spectrum. NIR light has a wavelength in the range 750-1400 nm, which is longer than visible light (380-750 nm). Human eyes cannot see NIR light but most digital cameras can sense it very well. NIR images of natural scenes usually have better contrast and contain rich texture details that may not be perceived in visible light photographs. Although NIR can be recorded by CCD or CMOS sensors, most manufacturers of digital cameras install an infra-red (IR) cutoff filter over the sensor to suppress infra-red light and avoid unwanted artifacts. NIR photography is commonly appreciated for its artistic value, but has not been fully exploited in computational photography. The contrast and rich texture details from a corresponding NIR image of a visible light image are useful for the visible light image quality enhancement.